Techniques for interacting with wearable devices

ABSTRACT

In one embodiment of the present invention, a gesture recognition application enables interactive entry via a touch pad. In operation, the gesture recognition application partitions the touch pad into multiple zones. Upon detecting a gesture via the touch pad, the gesture recognition application determines whether the gesture is zone-specific. If the gesture is zone-specific, then the gesture recognition application determines the zone based on the location of the gesture and then selects an input group based on the zone and the type of gesture. If the gesture is zone-agnostic, then the gesture recognition application selects an input group based on the type of gesture, irrespective of the location of the gesture. Advantageously, by providing zone-specific gesture recognition, the gesture recognition application increases the usability of touch pads with form factors that limit the type of gestures that can be efficiently performed via the touch pad.

BACKGROUND OF THE INVENTION

Field of the Invention

Embodiments of the present invention relate generally to computerprocessing and, more specifically, to techniques for interacting withwearable devices.

Description of the Related Art

Wearable devices, such as smart eyewear and heads-up displays, aredesigned to enable unobtrusive, constant, and immediate access toinformation. In general, wearable devices have small form factors thatsatisfy a variety of consumer requirements pertaining to comfort,aesthetics, and so forth. However, the small form factors typicallylimit the types of user-friendly interactions that the wearable devicessupport.

One approach to circumventing the limitations imposed by the small formfactor uses supplementary devices in conjunction with the wearabledevices. For example, a one-handed keyboard can be configured towirelessly communicate with smart eyewear. However, relying on suchsupplementary devices undermines one of the primary advantages ofwearable devices—unobtrusive access to information without theinconvenience associated with carrying or tethering to any device.

Another approach to bypassing the limitations imposed by the small formfactor employs voice-recognition techniques. However, while voice inputmay be useful in some situations, in other situations voice input is notan acceptable input mechanism. For example, in various situations,people often find a need or desire to relay private information towearable devices while in the presence of others.

Yet another approach to interacting with wearable devices leveragesexisting touch pads included in the wearable devices to enable limitedtactile interactions that can be reliably supported despite the smallform factor. For example, through a series of gestures, input can beconveyed to smart eyewear via a side touch pad. However, because thesize of the touch pads included in wearable devices is limited, manycommon input tasks, such as text entry, are challenging. For example,placing a standard QWERTY keyboard on the touch pad in a manner thatenables access to all of the characters is usually impractical.

As the foregoing illustrates, what is needed in the art are moreeffective techniques for interacting with wearable devices.

SUMMARY OF THE INVENTION

One embodiment of the present invention sets forth acomputer-implemented method for interacting with a wearable device. Themethod includes partitioning a touch pad associated with the wearabledevice into multiple zones; detecting a first gesture performed via thetouch pad; in response, determining whether a first gesture typeassociated with the first gesture is zone-specific; and if the firstgesture type is zone-specific, then: identifying a first zone includedin the multiple zones based on a first location associated with thefirst gesture, and setting a selected input group to equal a first inputgroup included in multiple input groups based the first zone and thefirst gesture type; or if the first type is not zone-specific, thensetting the selected input group to equal a second input group includedin the multiple input groups based on the first gesture type.

One advantage of the disclosed approach is that partitioning the sidetouch pad into multiple entry zones leverages the relatively widerdimension of the side touch pad to enable efficient gesture-based textentry for smart eyewear. Notably, by exploiting the side touch pads toprovide unobtrusive, constant access to information, the disclosedtechniques enable effective interaction with smart eyewear withoutcompromising the design goals of the smart eyewear.

BRIEF DESCRIPTION OF THE DRAWINGS

So that the manner in which the above recited features of the presentinvention can be understood in detail, a more particular description ofthe invention, briefly summarized above, may be had by reference toembodiments, some of which are illustrated in the appended drawings. Itis to be noted, however, that the appended drawings illustrate onlytypical embodiments of this invention and are therefore not to beconsidered limiting of its scope, for the invention may admit to otherequally effective embodiments.

FIG. 1 illustrates a smart eyewear system configured to implement one ormore aspects of the present invention;

FIG. 2 is a more detailed illustration of the computing device andgesture recognition application of FIG. 1, where the gesture recognitionapplication is configured to enable text input via the side touch pad,according to one embodiment of the present invention;

FIG. 3 is a conceptual diagram illustrating the results of hybrid entryoperations via the side touch pad of FIG. 1, according to variousembodiments of the present invention;

FIG. 4 is a conceptual diagram illustrating the results of zone-agnosticentry operations via the side touch pad of FIG. 1, according to variousembodiments of the present invention; and

FIG. 5 is a flow diagram of method steps for interacting with a wearabledevice, according to various embodiments of the present invention.

DETAILED DESCRIPTION

In the following description, numerous specific details are set forth toprovide a more thorough understanding of the present invention. However,it will be apparent to one of skill in the art that the presentinvention may be practiced without one or more of these specificdetails.

Smart Eyewear System Overview

FIG. 1 illustrates a smart eyewear system 100 configured to implementone or more aspects of the present invention. As shown, the smarteyewear system 100 includes, without limitation, a computing device 110,a display 190, and a side touch pad 150. The computing device 110, thedisplay 190, and the side touch pad 150 form an integrated unit referredto herein as “smart eyewear.” In alternate embodiments, the smarteyewear system 100 may be replaced with any system that includes anytype of wearable device and is configured to implement one or moreaspects of the present invention. Further, any number of the computingdevice 110, the side touch pad 150, and the display 190 may beimplemented independently of the other components, and any number of thecomponents may be mounted on the smart eyewear. In some embodiments,additional components may be mounted on the smart eyewear. For example,in some embodiments a camera may be integrated into the smart eyewear.

The computing device 100, the side touch pad 150, and the display 190may communicate in any technically feasible fashion. In someembodiments, any combination of the computing device 100, the side touchpad 150, and the display 190 may exchange data over one or more wiredconnections. In other embodiments, any combination of the computingdevice 100, the side touch pad 150, and the display may implement awireless communications protocol, such as Bluetooth. In yet otherembodiments, communication paths interconnecting any number of thecomputing device 110, the side touch pad 150, and the display 190 may beimplemented using any suitable protocols, such as PCI (PeripheralComponent Interconnect), PCI Express (PCI-E), AGP (Accelerated GraphicsPort), HyperTransport, or any other bus or point-to-point communicationprotocol(s), and connections between different components may usedifferent protocols, as is known in the art.

The smart eyewear system 100 may include any number of the computingdevices 110 configured with any number (including zero) of processingunits (CPUs and/or GPUs), memory units, etc. The computing device 110may be implemented in any technically feasible fashion. For example, thecomputing device 100 could be integrated with connection circuitry on asingle chip and mounted to the side of the smart eyewear. In anotherexample, the computing device 100 could be implemented within a smartphone and could be configured to communicate wirelessly with the smarteyewear.

In operation, the computing device 110 receives input user inputinformation from input devices, such as the side touch pad 150, issuescommands that control and coordinate the operations of the smart eyewearsystem 100, and transmits images to the display 190. The display 190 maybe any conventional cathode ray tube, liquid crystal display,light-emitting diode display, or the like. Notably, the display 190 istypically integrated into the smart eyewear in close proximity to thehead of the user. For example, the display 190 could be implemented asan optical head-mounted display, a heads-up display, and so forth.

The side touch pad 150 enables users to provide input to the smarteyewear system 100 using gestures that are typically performed using oneor more fingers. Such gestures may include swipes, taps, and the like.In general, a gesture recognition application 120 included in the smarteyewear system 100 and implemented within the computing device 110, mapsreceived gestures into other forms of input data that are amenable tocomputer processing, such as textual input. In particular, the gesturerecognition application 120 maps sequences of gestures entered via theside touch pad 150 to alphabetic characters. In alternate embodiments,the gesture recognition application 120 may perform any number and typeof operations that enable the gesture recognition application 120 totranslate one or more gestures into any form of input data.

Typically, to enable users to don and interact with the smart eyewear inan unobtrusive manner, the side touch pad 150 is implemented within arelatively small form factor that constrains the type of gestures thatare easily performed via the side touch pad 150. In particular, diagonalswipes are often difficult to execute properly via the side touch pad150. For this reason, the gesture recognition application 120 isconfigured to limit the number of types of meaningful gestures that maybe performed via the side touch pad 150. More specifically, the gesturerecognition application 120 is configured to map combinations ofvertical swipes, horizontal swipes, and taps to alphabetic charactersand to disregard other gestures, such as diagonal swipes. In general, asreferred to herein, the gesture recognition application 120 defines an“encoding” of gestures to items and then performs “gesture mapping”operations to translate gestures to corresponding items based on theencoding.

As persons skilled in the art will recognize, a naïve one-to-oneencoding of user-friendly gestures to characters may unacceptably limitthe number of unique characters that the gesture recognition application120 is configured to translate. For example, if the gesture recognitionapplication 120 were to implement a one-to-one encoding of the fiveacceptable gestures to characters, then the gesture recognitionapplication 120 would only support five characters. Accordingly, toprovide a unique encoding for each alphabetic character, the gesturerecognition application 120 is configured to encode each alphabeticcharacter into a sequence of two gestures and to distinguish verticalgestures based on the location of the vertical gesture within the sidetouch pad 150.

More specifically, the gesture recognition application 120 maps a firstgesture to a set of three characters, referred to herein as a “triplet,”and a second gesture to a specific character within the triplet. Tofacilitate location-based differentiation of gestures, the gesturerecognition application 120 divides the side touch pad 150 into threeseparate zones: a front zone 152, a middle zone 154, and a back zone156. As shown, because the width of the side touch pad 150 is longerthan the height of the side touch pad 150, to optimize gesture entry,each of the front zone 152, the middle zone 154, and the back zone 156spans a different portion of the width of the side touch pad 150.

In operation, the gesture recognition application differentiates betweenvertical gestures in a zone-specific manner. For instance, the gesturerecognition application 120 distinguishes between an upward gestureperformed within the front zone 152 and an upward gesture performedwithin the back zone 156. By contrast, the gesture recognitionapplication 120 interprets horizontal gestures and taps in a simpler,zone-agnostic manner. For instance, the gesture recognition application120 does not distinguish between a tap performed within the front zone152 and a tap performed within the back zone 156.

In alternate embodiments, the gesture recognition application 120 maydivide the side touch pad 150 into any number of sizes of zones. Ingeneral, the gesture recognition application 120 may consider any typeof gestures to be zone-agnostic and any type of gestures to bezone-specific in any combination and based on any number and type ofergonomic considerations. In some embodiments, to provide additionalguidance to the user, one or more of the back zone 156, the middle zone154, and the front zone 152 may include a tactile feature that enableseasy identification. For example, the middle zone 154 of the side touchpad 150 could include a strip of tape or could provide a differenttactile sensation than the back zone 156 and/or the front zone 152.

In alternate embodiments, the gesture recognition application 120 may beconfigured to accept any type of gestures in any combination and defineencodings based on the accepted gestures. For example, the gesturerecognition application could be configured to encode phone numbers asgestures entered using two fingers in unison. In some embodiments, uponreceiving unsupported gestures, the gesture recognition application 120is configured to ignore or perform error resolution operations insteadof attempting to map the gestures to characters.

Mapping Gestures

FIG. 2 is a more detailed illustration of the computing device 110 andthe gesture recognition application 120 of FIG. 1, where the gesturerecognition application 120 is configured to enable text input via theside touch pad 150, according to one embodiment of the presentinvention. As shown, the gesture recognition application 120 is includedin a memory unit 214 of the computing device 110 and executes on aprocessing unit 212 of the computing device 110. In alternateembodiments, the memory unit 214 may be replaced with an externalstorage unit, such as an optical storage device. The processing unit 212may be implemented in any technically feasible fashion and mayincorporate circuitry optimized for graphics and video processing,including, for example, video output circuitry.

As shown, the gesture recognition application 120 includes, withoutlimitation, a keyboard state 230, a zone 222, and a type 224. Thegesture recognition application 120 maintains the keyboard state 230 asa snap-shot in time of the mapping process. The keyboard state 230includes, without limitation, a selected triplet 240 and a selectedcharacter 250. Initially, the gesture recognition application 120 setsthe selected triplet 240 and the selected character 250 to reflect anunselected state (e.g., a NULL value) and transmits an initial keyboardstate 290 to the display 190. The initial keyboard state 290 is agraphical representation of the keyboard state 230 that prompts the userto begin the process of text entry via gestures.

The gesture recognition application 120 is configured to leverage auser's spatial memory of the common “QWERTY” keyboard layout to encodeeach alphabetic character into two gestures—a first gesture that selectsa triplet and a second gesture that selects a character within thetriplet. As part of implementing this triplet-based approach to textentry, the gesture recognition application 120 implements a modifiedQWERTY keyboard. To generate the modified QWERTY keyboard, the gesturerecognition application 120 partitions the alphabetic characters intotriplets in a uniform manner across three rows, based on the relativepositions of the characters included in the core rows of the standardQWERTY keyboard. In particular, the gesture recognition application 120shifts the locations of the characters “P” and “L” and includes a periodcharacter in the final triplet. In this fashion, the gesture recognitionapplication 120 maps the twenty-six alphabetic characters along with theperiod character into three rows, where each row includes threetriplets. To enable the user to easily perform the gestures required forsuccessful text entry, the initial keyboard state 290 reflects thelayout of the modified QWERTY keyboard.

Since diagonal swipes are relatively difficult to perform via the sidetouch pad 150, the gesture recognition application 120 does not includediagonal gestures in the encodings that the gesture recognitionapplication 120 defines for mapping a first gesture to the selectedtriplet 240. Instead, the gesture recognition application 120 relies onzone selection in conjunction with vertical gestures to define uniqueencodings for each triplet. In operation, if the first gesture is anupward gesture or a downward gesture, then the gesture recognitionapplication 120 maps the gesture to the selected triplet 240 based onthe zone of the first gesture (i.e., the zone in which the first gestureis performed). By contrast, if the first gesture is a forward swipe, abackward swipe, or a tap, then the gesture recognition application 120maps the first gesture to the selected triplet 240 without regard forthe zone(s) of the first gesture. The comprehensive, “hybrid” entryscheme defined by the gesture recognition application to unambiguouslymap each supported first gesture to a unique triplet is described ingreater detail below in the description of FIG. 3. In some embodiments,to emphasize the lack of diagonal gestures, the gesture recognitionapplication 120 highlights the corner triplets included in the initialkeyboard state 290.

After displaying the initial keyboard state 290 and receiving sensordata 215 corresponding to the first gesture from the side touch pad 150,the gesture recognition application 120 determines the type 224 of thefirst gesture based on the sensor data 215. The gesture recognitionapplication 120 may apply any combination of gesture recognitiontechniques to the sensor data 125 to classify the first gesture as anupward swipe, a downward swipe, a forward swipe, a backward swipe, or atap. If the gesture recognition application 120 determines that the type224 is an upward swipe or a downward swipe (i.e., a zone-specific firstgesture), then the gesture recognition application 120 determines thezone 222 of the first gesture based on the location of the first gestureas per the sensor data 215. The gesture recognition application 120 thenmaps the first gesture to the selected triplet 240 based on the type 224and the zone 222. By contrast, if the gesture recognition application120 determines that the type 222 is a forward swipe, a backward swipe,or a tap (i.e., a zone-agnostic first gesture), then the gesturerecognition application 120 maps the first gesture to the selectedtriplet 240 based on the type 224—without determining the zone 222. Inalternate embodiments, if the gesture recognition application 120determines that the gesture is zone-agnostic, then the gesturerecognition application 120 may set the zone 222 to a value thatindicates that the location of the first gesture is irrelevant.

After updating the selected triplet 240, the gesture recognitionapplication 120 transmits a graphical representation of the keyboardstate 230 to the display 190. The gesture recognition application 120may generate the graphical representation of the keyboard state 230 inany technically feasible fashion and may include any number of visualeffects to provide feedback to, prompt, and/or guide the user. Forexample, after updating the selected triplet 240, the gesturerecognition application 120 could generate a graphical representation ofmodified QWERTY keyboard layout that highlights the selected triplet240. Notably, the second gesture maps to one of the three charactersincluded in the selected triplet 240, and the horizontal layout of theselected triplet 240 intuitively maps to horizontal swipes and taps.Accordingly, the gesture recognition application 120 defines azone-agnostic entry process for the second gesture that limits theacceptable gestures to a forward swipe, a backward swipe, and a tap.

Upon receiving the sensor data 215 corresponding to a second gesture,the gesture recognition application 120 determines the type 224. Basedon the type 224, the gesture recognition application sets the selectedcharacter 250 to one of the three characters included in the selectedtriplet 240, thereby setting a final keyboard state. The comprehensive,zone-agnostic entry scheme defined by the gesture recognitionapplication 120 to unambiguously map each supported second gesture to aunique character within the selected triplet 240 is described in greaterdetail below in the description of FIG. 4.

The gesture recognition application 120 then transmits the selectedcharacter 250 to any client applications, generates a graphicalrepresentation of the keyboard state 230 that highlights the selectedcharacter 245, and transmits this graphical representation to thedisplay 190. Subsequently, to facilitate the entry of additionalcharacters, the gesture recognition application 120 sets the selectedtriplet 240 and the selected character 250 to reflect unselected states(e.g., NULL values) and transmits the initial keyboard state 290 to thedisplay 190.

In alternate embodiments, the gesture recognition application 120 may beconfigured set the keyboard state 230 based on other gestures and/oractions. For example, in some embodiments, the gesture recognitionapplication 120 may be configured to restore the keyboard to an initial,undefined state after detecting a two-second timeout. The gesturerecognition application 120 may encode other combinations of gesturesand/or actions to additional functions. For example, the gesturerecognition application 120 could map a double-swipe down-left to acharacter deletion operation, a double-swipe down-right to a spacecharacter, and/or a double-swipe up to trigger a switch to a symbols andnumbers keyboard.

FIG. 3 is a conceptual diagram illustrating the results of hybrid entryoperations 300 via the side touch pad 150 of FIG. 1, according tovarious embodiments of the present invention. As shown, nine firstgestures 325(1)-325(9) are depicted as entered via the side touch pad150. For each of the first gestures 325, a mapping to a selected triplet240 and a graphical representation of the resulting keyboard state 230,labelled as an intermediate keyboard state 390, are depicted.

In operation, upon receiving the sensor data 215 corresponding to thefirst gesture 325, the gesture recognition application 120 maps thefirst gesture 325 to the selected triplet 240 based on the type 224 ofthe first gesture 325. Further, if the type 224 of the first gesture 325is either an upward or a downward swipe, then the gesture recognitionapplication 120 maps the first gesture 325 to one of the triplets thatcorrespond to a corner of the initial keyboard state 290.

As shown, the first gesture 325(1) is an upward swipe that is enteredvia the front zone 152 included in the side touch pad 150. The gesturerecognition application 120 maps the first gesture 325(1) based on boththe type 224 and the zone 222 to the selected triplet 240(1) “QWE.”Subsequently, the gesture recognition application 120 generates theintermediate keyboard state 390(1) that includes the keyboard layoutwith the selected triplet 240(1) “QWE” highlighted.

The first gesture 325(2) is an upward swipe that is entered via themiddle zone 154 included in the side touch pad 150. The gesturerecognition application 120 maps the first gesture 325(2) based on boththe type 224 and the zone 222 to the selected triplet 240(2) “RTY.”Subsequently, the gesture recognition application 120 generates theintermediate keyboard state 390(2) that includes the keyboard layoutwith the selected triplet 240(2) “RTY” highlighted.

The first gesture 325(3) is an upward swipe that is entered via the backzone 156 included in the side touch pad 150. The gesture recognitionapplication 120 maps the first gesture 325(3) based on both the type 224and the zone 222 to the selected triplet 240(3) “UIO.” Subsequently, thegesture recognition application 120 generates the intermediate keyboardstate 390(3) that includes the keyboard layout with the selected triplet240(3) “UIO” highlighted.

As shown, the first gesture 325(4) is a forward swipe that is initiatedin any zone and may terminate in any zone included in the side touch pad150. For example, the forward swipe may initiate and terminate in theback zone 256, the forward swipe may initiate in the middle zone 254 andterminate in the front zone 252, and so forth. The gesture recognitionapplication 120 maps the first gesture 325(1) based the type 224“forward swipe”—irrespective of the zone(s)—to the selected triplet240(4) “ASD.” Subsequently, the gesture recognition application 120generates the intermediate keyboard state 390(4) that includes thekeyboard layout with the selected triplet 240(4) “ASD” highlighted.

The first gesture 325(5) is a tap that is entered in any zone includedin the side touch pad 150 and depicted as a filled circle. Forexplanatory purposes, some other first gestures 325 that are equivalentto the first gesture 325(5) with respect to the gesture recognitionapplication 120 are depicted as hollow circles. The gesture recognitionapplication 120 maps the first gesture 325(5) based the type 224“tap”—irrespective of the zone—to the selected triplet 240(5) “FGH.”Subsequently, the gesture recognition application 120 generates theintermediate keyboard state 390(5) that includes the keyboard layoutwith the selected triplet 240(5) “FGH” highlighted.

The first gesture 325(6) is a backward swipe that is initiated in anyzone and may terminate in any zone included in the side touch pad 150.For example, the backward swipe may initiate and terminate in the backzone 256, the backward swipe may initiate in the front zone 252 andterminate in the back zone 256, and so forth. The gesture recognitionapplication 120 maps the first gesture 325(6) based the type 224“backward swipe”—irrespective of the zone(s)—to the selected triplet240(6) “JKP.” Subsequently, the gesture recognition application 120generates the intermediate keyboard state 390(6) that includes thekeyboard layout with the selected triplet 240(6) “JKP” highlighted.

As shown, the first gesture 325(7) is a downward swipe that is enteredvia the front zone 152 included in the side touch pad 150. The gesturerecognition application 120 maps the first gesture 325(7) based on boththe type 224 and the zone 222 to the selected triplet 240(7) “ZXC.”Subsequently, the gesture recognition application 120 generates theintermediate keyboard state 390(7) that includes the keyboard layoutwith the selected triplet 240(7) “ZXC” highlighted.

The first gesture 325(8) is a downward swipe that is entered via themiddle zone 154 included in the side touch pad 150. The gesturerecognition application 120 maps the first gesture 325(8) based on boththe type 224 and the zone to the selected triplet 240(8) “VBN.”Subsequently, the gesture recognition application 120 generates theintermediate keyboard state 390(8) that includes the keyboard layoutwith the selected triplet 240(8) “VBN” highlighted.

The first gesture 325(9) is a downward swipe that is entered via theback zone 156 included in the side touch pad 150. The gesturerecognition application 120 maps the first gesture 325(9) based on boththe type 224 and the zone 222 to the selected triplet 240(9) “ML.”Subsequently, the gesture recognition application 120 generates theintermediate keyboard state 390(9) that includes the keyboard layoutwith the selected triplet 240(9) “ML” highlighted.

As FIG. 3 illustrates, the gesture recognition application 120 providesa user-intuitive and ergonomic set of first gestures 325 based on thehybrid entry 300 performed via the side touch pad 150. Notably, byleveraging different zones, the gesture recognition application 120eliminates the need for gestures such as diagonal swipes that arerelatively difficult to perform via the side touch pad 150 withoutlimiting the total number of triplets that may be selected via the setof first gestures 325.

FIG. 4 is a conceptual diagram illustrating the results of zone-agnosticentry 400 operations via the side touch pad 150 of FIG. 1, according tovarious embodiments of the present invention. For explanatory purposes,the context of FIG. 4 is that, the gesture recognition application 120maps the first gesture 325(1) to the selected triplet 240(1) “QWE” andthen generates the intermediate keyboard state 390(1) that includes thekeyboard layout with the selected triplet 240(1) “QWE” highlighted.

After transmitting the intermediate keyboard state 390(1) to the display190, the gesture recognition application 120 receives the sensor data215 corresponding to the second gesture 425. In operation, the sensordata 215 reflects that the first gesture 325 is followed by a particularzone-agnostic second gesture 425—a forward swipe 425(1), a tap 425(2),or a backward swipe 425(3). For explanatory purposes, the threemeaningful second gestures 425(1)-425(3) are depicted as entered via theside touch pad 150 and, for each of the second gestures 425, a mappingto a corresponding selected character 250 is depicted. Since there arethree characters included in the selected triplet 240, the gesturerecognition application 120 is configured to map three second gestures425 to the selected character 250 and to disregard any extraneous sensordata 215.

In alternate embodiments, the gesture recognition application 120 may beconfigured to perform any type of checking on the sensor data 215 andimplement any type of error resolution mechanisms, such as displaying an“invalid gesture” notification via the display 190. In general, thegesture recognition application 120 may perform any amount of errorchecking and/or resolution operations in conjunction with the hybridentry 300 associated with the first gesture 235 and/or the zone-agnosticentry 400 associated with the second gesture 425.

As shown, if the second gesture 425(1) is a forward swipe 425(1), thenthe gesture recognition application 120 maps the second gesture 425(1)to the selected character 250(1) “Q” based on the type 224 of the secondgesture 425(1) and the selected triplet 240(1). Notably, the gesturerecognition application 120 disregards the zone(s) in which the secondgesture 425(1) is performed while mapping the second gesture 425(1) tothe selected character 250(1).

If, however, the second gesture 425(2) is a tap, then the gesturerecognition application 120 maps the second gesture 425(2) to theselected character 250(2) “W” based on the type 224 of the secondgesture 425(2) and the selected triplet 240(1). The tap may be enteredin any zone included in the side touch pad 150 and is depicted as afilled circle. For explanatory purposes, some other second gestures 425that are equivalent to the second gesture 425(2) with respect to thegesture recognition application 120 are depicted as hollow circles.

If, however, the second gesture 425(3) is a backward swipe 425(3), thenthe gesture recognition application 120 maps the second gesture 425(3)to the selected character 250(3) “E” based on the type 224 of the secondgesture 425(3) and the selected triplet 240(1). Notably, the gesturerecognition application 120 disregards the zone(s) in which the secondgesture 425(3) is performed while mapping the second gesture 425(3) tothe selected character 250(3).

As shown, after determining the selected character 250, the gesturerecognition application 120 transmits the initial keyboard state 290 tothe display 190, thereby enabling the user to easily perform the nextfirst gesture 325. In general, the gesture recognition application 120may be configured to perform any number of operations designed toincrease the usability of the side touch pad 150 and guide the hybridentry 300 of the first gesture 235 and the zone-agnostic entry 400 ofthe second gesture 425. In particular, the gesture recognitionapplication 120 may be configured to provide any number and type ofvisual prompts via the display 190 at anytime. For example, in someembodiments, after mapping the second gesture 425, the gesturerecognition application 120 may transmit the selected character 250(0)“Q,” 250(1) “W,” or 250(2) “E” to the display 190 as an independentcharacter or as part of one or more characters (e.g., the selectedtriplet 240(1) or a larger portion of the keyboard).

As illustrated in FIGS. 3 and 4, by providing an intuitive set ofgestures for entry via the side touch pad 150 and visual prompts via thedisplay 190, the gesture recognition application 120 enables a user toefficiently convey textual input to the smart eyewear system 100. Inalternate embodiments, the gesture recognition application 120 may mapthe sensor data 215 in any technically feasible fashion to any form ofinput data. For example, in some embodiments, the gesture recognitionapplication 120 may be configured to reverse the mappings of the frontzone 152 and the back zone 156 (e.g., the gesture recognitionapplication 120 could map an upward swipe in the back zone 156 to theselected triplet 240 “QWE”). In yet other embodiments, the gesturerecognition application 120 may be configured to provide customizedmapping of any type and/or sequence of gestures to any type of input.For example, the gesture recognition application 120 could bedynamically configured to map a compound gesture that includes multipletaps and/or swipes in any number of zones to a particular Internetaddress.

Interacting with Smart Eyewear

FIG. 5 is a flow diagram of method steps for interacting with a wearabledevice, according to various embodiments of the present invention.Although the method steps are described with reference to the systems ofFIGS. 1-4, persons skilled in the art will understand that any systemconfigured to implement the method steps, in any order, falls within thescope of the present invention.

As shown, a method 500 begins at step 502, where the gesture recognitionapplication 120 partitions the side touch pad 150 into three zones: thefront zone 152, the middle zone 154, and the back zone 156. Morespecifically, because the height of the side touch pad 150 typicallyexceeds the width of the side touch pad 150, the gesture recognitionapplication 120 divides the horizontal span of the side touch pad 150into the front zone 152, the middle zone 154, and the back zone 156. Insome embodiments, to provide additional guidance to the user, one ormore of the front zone 152, the middle zone 154, and the back zone 156may include a tactile feature that enables easy identification. Forexample, the middle zone 154 of the side touch pad 150 may include astrip of tape or may provide a different tactile sensation than the backzone 156 and/or the front zone 152. In alternate embodiments, thegesture recognition application 120 may partition the side touch pad 150into any number of zones based on any number of ergonomicconsiderations. Further, the side touch pad 150 may include any type offeatures, including none, that may distinguish between the zones.

As part of step 502, the gesture recognition application 120 alsopartitions the keyboard characters into triplets, and then leverages thezones to map user-friendly gestures to the triplets. To exploit thefamiliar layout of the QWERTY keyboard, the gesture recognitionapplication 120 bases the mappings on the relative positions of thealphabetic characters within the three core rows of the QWERTY layout.To divide the alphabetic characters into triplets in a uniform manneracross the three rows, the gesture recognition application 120 shiftsthe locations of the characters “P” and “L” and includes a periodcharacter in the final triplet. In this fashion, the gesture recognitionapplication 120 maps the twenty-six alphabetic characters along with theperiod character into three rows, where each row includes threetriplets. Further, because diagonal swipes are relatively difficult toperform via the side touch pad 150, the gesture recognition application120 does not include diagonal gestures in the set of user-friendlygestures. Instead, the gesture recognition application relies on zoneselection to provide a set of user-friendly gestures that is rich enoughto support unique mappings for each triplet.

At step 504, the gesture recognition application 120 transmits theinitial keyboard state 290 to the display 190. In some embodiments, toguide the hybrid entry 300 associated with the first gesture 325, thegesture recognition application 120 highlights the triplets in eachcorner of the modified QWERTY keyboard (i.e., “QWE.” “UIO,” “ZXC,” and“ML”). In this fashion, the initial keyboard state 290 indirectlyemphasizes that selecting a highlighted triplet requires a verticalgestured in the front zone 152 or the back zone 156 of the side touchpad 150.

At step 506, the gesture recognition application 120 receives the sensordata 215 corresponding to the first gesture 325. The sensor data 215 mayinclude any amount and type of information related to the side touch pad190 that enables the gesture recognition application 120 to determinethe type 224 and the zone 222 of the first gesture 325. For example, thesensor data 215 could include starting and ending coordinates. At step508, the gesture recognition application 120 determines the type 224 ofthe first gesture 325. More specifically, the gesture recognitionapplication 120 applies one or more gesture recognition algorithms tothe sensor data 215 to classify the first gesture 325 as an upwardswipe, a downward swipe, a forward swipe, a backward swipe, or a tap.

At step 510, the gesture recognition application 120 determines whetherthe first gesture 325 is a vertical swipe (i.e., either an upward swipeor a downward swipe). If, at step 510, the gesture recognitionapplication 120 determines that the first gesture 325 is not a verticalswipe, then the gesture recognition application 120 considers the firstgesture 325 to be zone-agnostic, and the method 500 proceeds to step512. At step 512, the gesture recognition application 120 determines theselected triplet 240 based on the type 224 of the first gesture 325.Notably, the gesture recognition application 120 determines the selectedtriplet 240 without determining the zone(s) 222 associated with thefirst gesture 325. The method 500 then proceeds directly to step 518.

If, however, at step 510, the gesture recognition application 120determines that the first gesture 325 is a vertical swipe, then thegesture recognition application 120 considers the first gesture 325 tobe zone-specific, and the method 500 proceeds directly to step 514. Atstep 514, the gesture recognition application 120 determines the zone222 associated with the first gesture 325 based on the sensor data 215.More specifically, the gesture recognition application 120 sets the zone222 to one of the front zone 152, the middle zone 154, or the back zone156 based on the location of the first gesture 325. At step 516, thegesture recognition application 120 determines the selected triplet 240based on both the zone 222 and the type 224 of the first gesture 325.

At step 518, the gesture recognition application 120 generates theintermediate keyboard state 390 and transmits the intermediate keyboardstate 390 to the display 190. The intermediate keyboard state 390 mayinclude any type of feedback that distinguishes the selected triplet240. For example, the intermediate keyboard state 390 could include azoomed-in view of the selected triplet 240 superimposed on the entirekeyboard layout. Configuring the intermediate keyboard state 390 in sucha manner not only prompts the user to properly perform the secondgesture 425 to select the current character, but also assists the userin planning the next first gesture 325 to select the next triplet.

At step 520, the gesture recognition application 120 receives the sensordata 215 corresponding to the second gesture 425. The sensor data 215may include any amount and type of information related to the side touchpad 190 that enables the gesture recognition application 120 todetermine the type 224 of the second gesture 425. At step 522, thegesture recognition application 120 determines the type of the secondgesture 425. More specifically, the gesture recognition application 120applies one or more gesture recognition algorithms to the sensor data215 to classify the second gesture 425 as a forward swipe, a backwardswipe, or a tap.

At step 522, the gesture recognition application 120 determines, basedon the type 224 of the second gesture 425, which one of the threecharacters included in the selected triplet 240 is the selectedcharacter 250. At step 524, the gesture recognition application 120generates a final keyboard state with the selected character 250highlighted. As part of step 524, the gesture recognition application120 also conveys the selected character 250 to any number of clientapplications that receive text-based input via the gesture recognitionapplication 120. The method 500 then returns to step 504 where, tofacilitate the hybrid entry 300 of the next first gesture 325, thegesture recognition application 120 transmits the initial keyboard state290 to the display 190. The guest recognition application 120 continuesto cycle through steps 504-524, converting gesture-based input via theside touch pad 250 to the selected characters 250, and therebyfacilitating user-friendly interactions with the smart eyewear system100.

In sum, the disclosed techniques may be used to efficiently interactwith wearable devices. In particular, a gesture recognition applicationenables text entry via a side touch pad included in smart eyewear. Inoperation, the gesture recognition application divides the side touchpad horizontally into three zones. The gesture recognition applicationalso partitions the characters in a keyboard into nine triplets, whereeach triplet includes three characters. Guided by the location of thetriplets within a standard QWERTY keyboard, the gesture recognitionapplication then associates the top row of triplets with zone-specificupward swipes and the bottom row of triplets with zone-specific downwardswipes. Further, the gesture recognition application associates themiddle row of triplets with, from left to right across the QWERTYkeyboard: a zone-agnostic forward swipe, a zone-agnostic tap, and azone-agnostic backward swipe. To facilitate text entry, the gesturerecognition application configures the smart eyewear to display aninitial state of the keyboard delineated by the triplets.

Upon detecting a first gesture via the side touch pad, the gesturerecognition application determines whether the first gesture is avertical gesture. If the first gesture is a vertical gesture, then thegesture recognition application determines the zone of the first gesturebased on the location of the initial gesture. The gesture recognitionapplication then selects a triplet based on the zone and the type(either an upward or downward swipe) of the first gesture. If the firstgesture is not a vertical gesture, then the gesture recognitionapplication selects the triplet based on the type of the firstgesture—the location of the first gesture is irrelevant. The gesturerecognition application then configures the smart eyewear to display theselected triplet within the keyboard.

Upon detecting a second gesture via the side touch pad, the gesturerecognition application selects one of the three characters included inthe selected triplet. Notably, the second gesture is a zone-agnosticforward swipe, a zone-agnostic backward swipe, or a zone-agnostic tap.The gesture recognition application then configures the smart eyewear toredisplay the keyboard in the initial state to facilitate entry of thenext character.

Advantageously, partitioning the side touch pad into multiple entryzones enables efficient gesture-based text entry for smart eyewear. Morespecifically, the zone-based techniques disclosed herein leverage therelatively wider dimension of the side touch pad to provide acomprehensive set of mappings between ergonomic gestures and triplets.In particular, because diagonal swipes are relatively difficult toproperty execute on the side touch pad, the gesture recognitionapplication maps zone-specific vertical swipes to triplets that arelocated at the diagonal corners of the keyboard. Notably, the gesturerecognition application exploits the side touch pad to providetext-based input without compromising the primary purpose of the smarteyewear—to provide unobtrusive, constant access to information.

The descriptions of the various embodiments have been presented forpurposes of illustration, but are not intended to be exhaustive orlimited to the embodiments disclosed. Many modifications and variationswill be apparent to those of ordinary skill in the art without departingfrom the scope and spirit of the described embodiments.

Aspects of the present embodiments may be embodied as a system, methodor computer program product. Accordingly, aspects of the presentdisclosure may take the form of an entirely hardware embodiment, anentirely software embodiment (including firmware, resident software,micro-code, etc.) or an embodiment combining software and hardwareaspects that may all generally be referred to herein as a “circuit,”“module” or “system.” Furthermore, aspects of the present disclosure maytake the form of a computer program product embodied in one or morecomputer readable medium(s) having computer readable program codeembodied thereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an optical fiber,a portable compact disc read-only memory (CD-ROM), an optical storagedevice, a magnetic storage device, or any suitable combination of theforegoing. In the context of this document, a computer readable storagemedium may be any tangible medium that can contain, or store a programfor use by or in connection with an instruction execution system,apparatus, or device.

Aspects of the present disclosure are described above with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of thedisclosure. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, enable the implementation of the functions/acts specified inthe flowchart and/or block diagram block or blocks. Such processors maybe, without limitation, general purpose processors, special-purposeprocessors, application-specific processors, or field-programmableprocessors or gate arrays.

The flowchart and block diagrams in the figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present disclosure. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

While the preceding is directed to embodiments of the presentdisclosure, other and further embodiments of the disclosure may bedevised without departing from the basic scope thereof, and the scopethereof is determined by the claims that follow.

What is claimed is:
 1. A computer-implemented method for interactingwith a wearable device, the method comprising: partitioning a touch padassociated with the wearable device into a plurality of zones; detectinga first gesture performed via the touch pad; in response, determiningwhether a first gesture type associated with the first gesture iszone-specific; and if the first gesture type is zone-specific, then:identifying a first zone included in the plurality of zones based on afirst location associated with the first gesture, and setting a selectedinput group to equal a first input group included in a plurality ofinput groups based on the first zone and the first gesture type; and ifthe first gesture type is not zone-specific, then setting the selectedinput group to equal a second input group included in the plurality ofinput groups based on the first gesture type.
 2. The method of claim 1,further comprising detecting a second gesture performed via the touchpad and, in response, selecting a first item included in the selectedinput group.
 3. The method of claim 2, wherein the selected input groupcomprises a plurality of characters, and the first item comprises afirst character.
 4. The method of claim 1, wherein determining whetherthe first gesture type is zone-specific comprises determining whetherthe first gesture type comprises a vertical swipe.
 5. The method ofclaim 4, wherein the first gesture type comprises a vertical swipe, andsetting the selected input group comprises selecting an input group thatis located in a diagonal direction with respect to a center input groupincluded in the plurality of input groups.
 6. The method of claim 1,wherein the selected input group comprises one or more characters, andfurther comprising displaying via a display associated with the wearabledevice at least a portion of a keyboard that includes the selected inputgroup.
 7. The method of claim 1, wherein the first gesture comprises anupward swipe, a downward swipe, a forward swipe, a backward swipe, or atap.
 8. The method of claim 1, wherein a height associated with thetouch pad is less than a width associated with the touch pad, and eachzone in the plurality of zones spans the height associated with thetouch pad and spans a different portion of the width associated with thetouch pad.
 9. A non-transitory computer-readable storage mediumincluding instructions that, when executed by a processing unit, causethe processing unit to interact with a wearable device by performing thesteps of: partitioning a touch pad associated with the wearable deviceinto a plurality of zones; detecting a first gesture performed via thetouch pad; in response, determining whether a first gesture typeassociated with the first gesture is zone-specific; and if the firstgesture type is zone-specific, then: identifying a first zone includedin the plurality of zones based on a first location associated with thefirst gesture, and setting a selected input group to equal a first inputgroup included in a plurality of input groups based on the first zoneand the first gesture type; and if the first gesture type is notzone-specific, then setting the selected input group to equal a secondinput group included in the plurality of input groups based on the firstgesture type.
 10. The non-transitory computer-readable storage medium ofclaim 9, further comprising detecting a second gesture performed via thetouch pad and, in response, selecting a first item included in theselected input group.
 11. The non-transitory computer-readable storagemedium of claim 10, wherein the selected input group comprises aplurality of characters, and the first item comprises a first character.12. The non-transitory computer-readable storage medium of claim 9,wherein the plurality of input groups comprises nine input groups, andthe first input group comprises three characters.
 13. The non-transitorycomputer-readable storage medium of claim 9, wherein determining whetherthe first gesture type is zone-specific comprises determining whetherthe first gesture type comprises a vertical swipe.
 14. Thenon-transitory computer-readable storage medium of claim 9, wherein thefirst gesture comprises an upward swipe, a downward swipe, a forwardswipe, a backward swipe, or a tap.
 15. The non-transitorycomputer-readable storage medium of claim 9, wherein the first gesturecomprises at least two of an upward swipe, a downward swipe, a forwardswipe, a backward swipe, and a tap.
 16. A system configured to interactwith a wearable device, the system comprising: a memory storing agesture recognition application; and a processing unit coupled to thememory, wherein, when executed by the processor, the gesture recognitionapplication configures the processor to: partition a touch padassociated with the wearable device into a plurality of zones; detect afirst gesture performed via the touch pad; in response, determinewhether a first gesture type associated with the first gesture iszone-specific; and if the first gesture type is zone-specific, then:identify a first zone included in the plurality of zones based on afirst location associated with the first gesture, and set a selectedinput group to equal a first input group included in a plurality ofinput groups based on the first zone and the first gesture type; and ifthe first gesture type is not zone-specific, then set the selected inputgroup to equal a second input group included in the plurality of inputgroups based on the first gesture type.
 17. The system of claim 16,wherein the gesture recognition application further configures theprocessing unit to detect a second gesture performed via the touch padand, in response, select a first item included in the selected inputgroup.
 18. The system of claim 17, wherein the selected input groupcomprises a plurality of characters, and the first item comprises afirst character.
 19. The system of claim 16, wherein a height associatedwith the touch pad is less than a width associated with the touch pad,and each zone in the plurality of zones spans the height associated withthe touch pad and spans a different portion of the width associated withthe touch pad.
 20. The system of claim 16, wherein the plurality ofzones further includes a second zone, and a tactile sensation associatedwith the first zone is different than a tactile sensation associatedwith the second zone.